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ABSTRACT 


A new stable and efficient implementation of the Lanczos algorithm is 
presented. 


The Lanczos algorithm is a powerful method for finding a few eigenvalues 
and eigenvectors at one or both ends of the spectrum of a symmetric matrix A. 
The algorithm is particularly effective if A is large and sparse in that the 
only way in which A enters the calculation is through a subroutine which com- 
putes Av for any vector v. Thus the user is free to take advantage of any 
sparsity structure in A and A need not even be represented as a matrix at all. 


The simple Lanczos algorithm procedes as follows. Choose q , an arbitrary 
unit vector, and define 3^ = 0 and = 0. Then for j = 1,2,... do 1 to 5- 
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One cycle through 1-5 is a Lanczos step. Note that only c Lj_q and q^ are needed 
to compute which is another attractive feature of the algorithm. 

In exact arithmetic, if Qj = (dp 5G2 ’ * • ’Gj ) then it can be shown (cf. 
ref. l) that Q. is an orthonormal maurix, i.e. 1-Q*Q. = 0, and in addition 

Q*AQ. = T. where 3 3 
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is tridiagonal. 


Furthermore if T . = S.0.S. is the spectral decomposition of 
J J J J 

T ; with 0, = diag(e ( ' j) ,0^ ) ,...,e ( ' j) ) and if Y = (y ( j } ,y^ J } , - . . ,y ( . 3 } ) = Q S 
/ j j ^ ^ 0 J X £. j j J 

,y) d; ), i = l,2,...,j, are the (optimal) Rayleigh-Ritz approximations 


then (0 : 
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to eigenpairs of A derivable from span(Qj), the subspace spanned by 


Q.2_ 5 ^2 ’ ’ ’ 1 


q j 


Finally and remarkably, the residual norm of (y.,6.) can be computed with- 
out computing the vector y . Namely 1 


II Ay . -y. 0 . II 

ill 



where = B j 1 s j p I and s ji is t ^ ie (jji) element of S j . The quantities 3-^ 


out the appearance of a small 3 . • 

even if 6. is not small at all. J 
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Ji 


show how it is possible for some of the Ritz values (9's) to be accurate vi th- 

ey' 


If s.. is tiny then 0) 
Ji i 


will be accurate 


By construction span(Qj ) = span(q^ ,Aq^ 5 • • • ,A J qp), a Krylov subspace. It 
can be shown (refs. 2 and 3) that Rayleigh-Ritz approximations converge rapidly 
as j increases to well separated extreme eigenpairs of A (those near either end 
of the spectrum) . 


Unfortunately, as was known to Lanczos when he introduced the algorithm 
(ref. U), finite precision causes the computed quantities to diverge completely 
from their theoretical counterparts. The Lanczos vectors (the q's) inevitably 
lose their mutual orthogonality and approach linear dependence. This is the 
infamous "loss of orthogonality" in the Lanczos algorithm. 

Lanczos himself recommended that the simple Lanczos algorithm be augmented 
by a full reorthogonalization of each newly computed q. . That is, q. + ^ is 
explicitly orthogonalized against all preceding Lanczoi vectors (q. , fir i j ) . 
This not only greatly increases the number of computations required to compute 
q^ + ^, it also requires that all the q's be kept in fast store. 

This poses a serious dilemma. For large problems it will be too costly to 
take more than a few steps using full reorthogonalization but linear indepen- 
dence will surely be lost without some sort of corrective procedure. Selective 
Orthogonalization (hereafter called SO) interpolates between full reorthogo- 
nalization and simple Lanczos to obtain the best of both worlds. Robust linear 
independence is maintained among the columns of Q. at a cost which is close to 
that of simple Lanczos. ^ 
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SO is "based on the analysis of the simple Lanczos algorithm in finite 
arithmetic given "by C. Paige in his doctoral thesis (ref. 3). Paige showed 
that the loss of orthogonality among the columns of is highly structured 
when viewed in the "basis of Ritz vectors, the columns of Y . = Q.S., rather 

than in the basis of Q. itself. J J J 

J 


Theorem (Paige). For any step j of the simple Lanczos algorithm and 

any ill f j | 

|y UJ *q. +1 ! - 6 II All y . . / B . . 

' i J+l Ji Ji 

where y_ = 1 and 6 is the working precision. 


* 

A proof is given in reference 5- 

It can also be shown (cf. ref. 5 or 6) that is a very good estimate of 
the residual norm of y ^ ' despite rounding errors. Thus Paige's Theorem shows 
that will lose orthogonality only in the direction of Ritz vectors with 

small that is those Ritz vectors which are converging to eigenvectors. 

This can be stated as 


loss of orthogonality ^ convergence 


To maintain orthogonality among the Lanczos vectors below some threshold 
value T £ 1 it is only necessary 
tors which satisfy 

y 

By Paige's Theorem equation (l) holds only if Bji £ ellAllyj^/T = e II All / t . Thus 
it is possible to determine which Ritz vectors achieve the threshold (l) merely 
by inspecting the 6j^ which can be computed via a small (j x j) eigenproblem. 
There are strong theoretical arguments in favor of the choice T = /e (ref. 5 
or 6) . 


to orthogonalize q., n against those Ritz vec- 

J + l 


(j)* 

i 


Vi 1 * T 


(1) 


Thus SO modifies the simple Lanczos algorithm by explicitly orthogonaliz- 
ing q. against all the Ritz vectors which satisfy 


S.. < /ell All 
Ji - 


(2) 


We call any Ritz vector which satisfies (2) a good Ritz vector. Good Ritz vec- 
tors are already rather well converged and few (if any) of the Ritz vectors at 
step j will be good, which explains the computational efficiency of the scheme. 


In practice it is possible to implement SO even more efficiently. It is 
not necessary to recompute the good Ritz vectors for orthogonalizing lj + ]_ a "t 
each step j as Ritz vectors computed at earlier steps can be used instead. 

% 

The detailed rounding error analysis needed to complete the proof of Paige's 
Theorem is given in reference 3- 
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Furthermore it is not necessary to orthogonalize against a particular good 
Ritz vector at every step. 

In conclusion, SO is an efficient way to maintain robust linear indepen- 
dence among the columns of Qj and so allow the Lanczos algorithm to he run 
almost as originally conceived. SO points the way to a high quality subroutine 
package which can be used off the shelf for large sparse symmetric eigenvalue 
problems . 
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